Skip to main content

View Post [edit]

Poster: molly Date: Mar 29, 2005 7:00am
Forum: web Subject: Re: Robots archive - noarchive META tags

The Wayback Machine does adhere to properly formatted NoArchive META tags, but only for specific documents that include this tag. If the tag was added in 2002, pages before that will still appear. If you think otherwise, and would like to bring a specific example to our attention, please send an archival URL (http://web.archive.org/TIMESTAMP/URL) to info AT archive.org.

To block specific UserAgents while allowing access to others, robots.txt is a good choice. This will also allow you to control robot access from a central location instead of managing this tag in a large number of documents.

Thanks for using our services.